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Using the experimental television facility described in a companion 
paper, frame repeating and point-by-point selective replenishment of pic- 
ture elements have been accomplished in real time. On the basis of initial 
experiments, using the head-and-shoulder view of a person as the picture 
source, such as is likely to be encountered in a visual communication system, 
the following tentative conclusions have been reached: 

(i) The motion rendition with a 15 new pictures /second frame repeal- 
ing system, while not flawless, is reasonably good. 

(ii) Selectively replenishing one-quarter of the picture points per frame 
gives a better continuity of motion but results in objectionable 
patterns. 

(Hi) Picture quality greatly depends on the pattern of picture replenish- 
ment. Of the five replenishment patterns tested, two result in pictures 
which are significantly better than the other three. 

(iv) In informal viewings, opinion has been so divided that no preference 
has been established between simple 15 new pictures/ second frame 
repeating and the more satisfying schemes for picture replenish- 
ment. 

(v) The frame repeating and replenishment systems produce gross 
impairment during zooming and panning; consequently, these 
systems in their present form are unlikely to be useful for broadcast 
television. 

The impairments observed in these systems are subjective and not yet 
predictable. This emphasizes the importance of subjective testing of systems 
in real time. 
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I. INTRODUCTION 

Experiments on real-time operation of various frame repeating and 
replenishment systems were performed with a low-resolution TV sys- 
tem described in a companion paper. 1 The head-and-shoulder view 
of a person was used as the picture source as might be used in a visual 
communication system. 

As is well-known, the lower bound on the number of presentations 
per second in a television system is set by the flicker requirements. No 
systematic study of motion in TV pictures has been possible in the 
past because the flicker requirements dictate that for a screen lumi- 
nance of, say, 85 cd/m 2 (25 fL) , something of the order of 60 pres- 
entations must be presented every second to alleviate the flicker 
problem. The presentation requirements for producing smooth per- 
ceived motion for most of the common types of movements are much 
less than this. In the past, suitable means have not been available 
for the study of the lower bound on the picture rate necessary for the 
satisfactory rendition of motion in television systems. 

With the availability of large-capacity, high-speed ultrasonic delay 
lines, 2 it is now possible to store, in digital form, large quantities of 
information. Delay lines with a total storage capacity of 25 kilobits 
and an input-output rate of approximately 1.5 megabits/second are 
readily available now. Using eight assemblies of lines and a picture 
format of 160 samples per line, 160 lines per frame sequentially 
scanned at 60 frames per second, allows storage of one complete pic- 
ture frame encoded as 8-digit PCM at a sampling rate of 1.536 MHz. 

The stored information can be displayed several times on the dis- 
play monitor at a suitable rate to satisfy the requirements for display 
flicker. This allows sending new information through the channel to 
satisfy only the condition of suitable rendition of motion. The avail- 
ability of frame storage has made it possible to study motion rendi- 
tion separated from display flicker effects. 

The rendition of motion by rapid superimposition of a sequence of 
still pictures is based on the exploitation of a perceptual property of 
the viewer, namely, the persistence of vision. Our knowledge of the 
human perceptual mechanism is still quite incomplete and primitive. 
Hence, it is obvious that a system exploiting perceptual redundancy 
can only be evaluated by obtaining the subjective responses of the 
viewers in controlled experiments. To obtain the subjective responses, 
the operation or simulation of the systems on the time scale identical 
to the original motion is imperative and its importance cannot be over- 
emphasized. 
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Frame repetition is taken to mean that one complete frame is 
transmitted to the receiver, stored in the memory and then shown 
repetitively a number of times on the TV screen. In a replenishment 
system a portion of the picture information from each frame is 
transmitted during that frame time. At the receiver this information 
must be inserted in the memory in the appropriate time slots. Com- 
plete pictures are read out of the memory each frame time for display, 
but the picture changes from frame to frame as the new information 
is added. 

II. SYSTEM DESCRIPTION 

A block diagram of the basic system is shown in Fig. 1. The details 
of the system's components are described elsewhere. 1 The video signal 
is generated by a vidicon camera system designed to sequentially scan 
the vidicon at 60 frames per second with a square raster of 160 lines. 
Suitable low-pass filtering limits the bandwidth of the signal to less 
than 768 kHz. Unless otherwise stated, the picture was restricted to a 
head-and-shoulder view of a person. 

The video signal is sampled at the rate of 1.536 MHz and the 
samples are encoded by an 8-digit PCM encoder. The output digits of 
the encoder are in parallel form. Each digit is routed to a 1/60 second 
delay line. The switch S in Fig. 1, at any instant, connects each delay 
line input either to the respective encoder output or to its own output. 
If the switch is at position A, new data is inserted into each delay line 
and is also decoded and displayed. If the switch is at position B, the 
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Fig. 1 — Basic frame repeating and replenishment systems. 
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data, which has been previously stored and displayed, is displayed 
again and is recirculated through the delay lines. Whatever the 
switch position, the monitor display rate remains constant at 60 
frames/second. The switch S operates fast enough that any selected 
sample or group of samples can be replaced. 

By generating different switch control waveforms, we can make 
the basic system operate as different frame repeating or replenishment 
systems. By constraining switch S to remain at position B, we can 
operate the memory in the circulating mode and store its contents 
indefinitely. This feature allows close scrutiny of any particular frame 
and also eases the problem of taking still photographs. 

While the subject is producing exaggerated hand and head motion, 
the memory may be placed in the circulating mode. This ensures that 
a single frame is frozen in the memory and is available for photo- 
graphing. This procedure has been followed for taking all the photo- 
graphs included in this paper. 

Observations were conducted in the laboratory at a reduced room 
illumination of approximately 300 lux (30 fc). The monitor was 
adjusted to have a highlight luminance of approximately 340 cd/m 2 
(100 fL). 

2.1 Frame Repeating Systems 

The basic frame repeating principle can be illustrated by Fig. 2. 
The vidicon is sequentially scanned every 1/60 second. However, only 
every nth frame is displayed on the monitor and stored in the memory. 
This frame is then repeated (n — 1) times on the monitor. The pres- 
entation rate is maintained at 60 frames per second to combat flicker. 
In other words, every nth frame is sent to the receiver which uses 
zero-order interpolation to fill in the missing frames. 

In the basic experiment described above, the camera is sequentially 
scanned every 1/60 second. Switch S' is open. Thus, the visual signal 
is integrated on the camera photo-cathode for 1/60 second. The prob- 
lem of the optimum integration period will be discussed later. Switch 
S" from the noise generator is open for these experiments. 

2.2 Replenishment Systems 

In a replenishment system, only a fraction of the total picture 
information is replenished during each frame period. Using signal 
storage, replenishment systems build up a picture by sending \/n of 
the total number of samples contained in a frame in one frame period. 
Thereafter, in every presentation, 1/n of the samples are new while 
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the rest are old.* Thus, new samples continually replenish old ones 
according to some predetermined scheme. 



III. SOME RESULTS 



Several frame repeating and replenishment systems have been 
operated in real time. Impairments are produced only in those areas 
where there is real or apparent movement, i.e., where the luminance at 
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Fig. 2 — Frame repeating. 



a point changes from frame-to-frame. Further, the type of impairment 
for a particular system depends on the type of motion. The impair- 
ments introduced in motion rendition are subjective and are extremely 
difficult to describe. However, these will be described as best as pos- 
sible. Photographs will be used to illustrate the effects wherever 
possible. As all the impairments are produced only in the presence of 
motion, photographs are not very satisfactory. Fig. 3(a) and 4(a) 
show a picture when no frame repeating or replenishment is used. 

3.1 Frame Repeating Systems 

Using a vidicon integration time of 1/60 second, frame repeating 
was observed using rates of 30, 20, 15, 12, 10, 8.6, 7.5, 6.7, and 6 new 
pictures/second. 



* Thia assumes replenishment of ecpial number of samples eveiy frame time. 
This assumption is not necessary in a general case. 
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As we go from higher rates to lower rates, the impairment, which 
appears in the form of a jerky motion, increases. For the head-and- 
shoulder view of a person conversing, no impairment is noted at 30 
new pictures/second. At 20 new pictures/second, fast large-area mo- 
tion produces a slight amount of jerkiness. At 15 new pictures/second, 
noticeable jerkiness is observed for large-area motion, such as head 
motion. However, the lip motion still appears natural. At 12 new pic- 
tures/second, the jerkiness is quite pronounced and some discrepancy 
can be noticed in the lip region also. As we go down to lower rates, 
further gradual increase in jerkiness occurs. At 6 new pictures/second 
the picture is quite jerky. The coordination between the lip motion and 
the sound is almost completely lacking. 

Responses of several engineers after informal viewing suggests that 
the threshold of acceptability may be between 12 and 15 new pictures/ 
second. The change in quality as we go from 15 new pictures/second 
to 12 new pictures/second seems more drastic than for any other pair 
of adjacent rates. It is proposed to check this observation by careful 
subjective experiments. 

In another experiment, a broadcast TV program was displayed on a 
TV receiver and was used as the picture source for the system's 
vidicon camera. Everything else remained the same. It was observed 
that frame repeating produced gross impairment during zooming and 
panning of the broadcast studio camera which is quite common in 
entertainment TV programs. 

3.2 Replenishment Systems 

The following replenishment systems have been observed in real 
time : 

3.2.1 2 : 1 Replenishment. 

The replenishment pattern is shown in Fig. 3(b). The camera was 
scanned sequentially at the rate of 60 frames/second. The samples 
marked 1 were replenished in frame 1 and the samples marked 2 were 
replenished in frame 2. Thus, the total replenishment time was 1/30 
second. A still photograph depicting the system output is shown in 
Fig. 3(b). 

Very slight impairment in the rendition of motion was observed for 
this condition. 

3.2.2 4 : 1 Replenishment. 

The camera was scanned sequentially at the rate of 60 frames/second. 
Samples were replenished according to five different replenishment 
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Fig. 3 — Photographs illustrating replenishment patterns. 
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patterns to be described. In each case, the sample positions marked 1 
were replenished in frame 1, 2 in frame 2, 3 in frame 3, and 4 in frame 
4. Total replenishment time was 1/15 second. 

3.2.2.1 Pattern 1. The replenishment pattern is shown in Fig. 3(c). 
The replenishment pattern is vertical. Vertical bars were quite visible 
in the presence of motion. The impairment was substantial. Fig. 3(c) 
shows a still photograph depicting the output of the system in the 
presence of exaggerated head motion. 

3.2.2.2 Pattern. 2. The replenishment pattern is as shown in Fig. 3(d). 
As the replenishment pattern is diagonal, the visible bars in the 
photograph (Fig. 3(d)), are diagonal. In real time observations, these 
are less visible than the vertical bars in pattern 1. However, the pat- 
terns were still visible enough to be objectionable. 

3.2.2.3 Pattern S. The replenishment pattern is as shown in Fig. 4(b). 
As can be seen by an inspection of the replenishment pattern, simple 
vertical or diagonal patterns are nonexistent for the replenished 
points. This system performs much better than the two previously 
described replenishment systems of corresponding rate. Moderate and 
fast motions do cause some "ragged"-edge effects. 

3.2.2.4 Pattern 4- Another replenishment pattern is shown in Fig. 
4(c). This figure also shows a still photograph depicting the output of 
the system in the presence of exaggerated head motion. The perform- 
ance of this system is similar to that of the system described in Section 
3.2.2.3 above. 

3.2.2.5 Pattern 5. The pseudo-random pattern. In this system, the 
average bit rate is the same as in the other 4 : 1 replenishment systems 
described above. However, the pattern of replenishment is not regular 
over blocks of 4 picture elements but is pseudo-random. One quarter 
of the total number of samples stored in the memory is replenished, in 
a random pattern, each TV frame period. All information stored in 
the memory is replenished once and only once during any four con- 
secutive TV frame periods. The same replenishment pattern is re- 
peated every four TV frame periods. Fig. 4(d) shows a still photo- 
graph depicting the output of the system in the presence of exaggerated 
head motion. The system performance is not satisfactory. Moving 
edges break up and produce a sparkling effect. 
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3.3 Frame Repeating With Increased Vidicon Integration Time 

The vidicon is a storage type of camera tube. In the standard 
sequential operation, the light distribution of the scene is integrated 
on the camera photosensitive element for 1/60 second. For fast motions 
this integration produces a blur. This effect can be contrasted with a 
shuttered or a nonstorage type of camera unit that would have pro- 
duced jerkiness in such a situation. It is felt that a trade-off between 
sharp but jerky pictures and blurred but smooth pictures exists and 
there may be a subjectively optimum integration time and charac- 
teristic for different uses of television. 

In the frame repeating experiments described in Section 3.1, the 
integration time was 1/60 second. On the other hand, if the camera 
is read out only when a new picture is desired, the image will be 
integrated on the camera photosensitive element for the full n/60 
second if the system uses 60/n new pictures/second. Possibilities exist 
to obtain any arbitrary integration characteristic if we use external 
storage. 

A preliminary experiment was tried using the vidicon itself as the 
integration element. No conclusive results were obtained. The Switch 
S' in Fig. 1 is closed to pass a gating signal to the vidicon to read out 
the integrated picture every fourth frame time for storage in the 
memory. However, the nonlinear brightness-current characteristic 
of the vidicon, the increased stickiness, and the effect of lateral leak- 
age in a vidicon system with increased integration time could have 
contributed more detrimentally to the resulting display than the im- 
provement from the anticipated integration characteristic. 

3.4 Frame Repeating With Noisy Signal 

The effect of frame repeating on the visibility of additive noise was 
observed. The switch S" in Fig. 1 was closed to add noise to the picture 
signal. 

For a reduced number of pictures per second, the noise pattern 
remained frozen and then suddenly changed to another pattern as a 
new picture was read in. The noisy picture lost its "busy" look to a 
certain extent. The impression gained was one of a superimposed 
noise pattern moving jerkily. The visibility of the noise increased as 
the number of new pictures/second was decreased. 

IV. DISCUSSION 

The impairments produced by the frame repeating systems and the 
replenishment systems are very different. 
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In the frame repeating systems, the impairments are in the form 
of jerky motions. These are most visible where the moving edges are 
the outlining contours of large areas. There also seems to be a correla- 
tion between the average brightness of the area and the visibility of 
jerkiness. It is an apparent flicker effect. 

In the replenishment systems, the jerkiness is missing. However, 
the impairment appears in the form of ragged edges of the moving 
objects and the superimposition of visible patterns. The replenishment 
experiments described clearly bring out the importance of operation 
or simulation of systems on the time scale identical to the original 
motion for the purpose of subjective evaluation. It may be recalled 
that for the same bit rate, the performance of the replenishment 
systems varied widely depending on slight changes in the replenish- 
ment patterns. On the basis of observations made so far, there seems 
to exist a correlation between the visibility of undesired patterns and 
the spacing of the points replenished in the same frame. For example, 
in Fig. 3(c), all the points replenished in any particular frame, say 
those designated by 1, are in a vertical column, and this produces a 
vertical bar pattern. In Fig. 3(d), all the points replenished in frame 
1 are along a diagonal. In this case, we see spurious diagonal bar 
patterns. 

In Figs. 4(b) and (c), all the picture points designated 1 are 
separated by only one sample of another frame in both horizontal and 
vertical directions. Of the five cases studied, patterns are least visible 
in these cases. It is interesting to note that the pseudo-random re- 
plenishment pattern did not perform as well. 
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